In this work we introduce Quixote, a system that makes programming virtual agents more accessible to non-programmers by enabling these agents to be trained using the sociocultural knowledge present in stories. Quixote uses a corpus of exemplar stories to automatically engineer a reward function that is used to train virtual agents to exhibit desired behaviors using reinforcement learning. We show the effectiveness of our system with a case study conducted in a virtual environment called Robbery World that simulates a bank robbery scenario. In this case study, we use a corpus of stories crowdsourced from Amazon Mechanical Turk to guide learning. We evaluate Quixote under a variety of different conditions to determine the overall effectiveness of the system in Robbery World.

Computational systems can use sociocultural knowledge to understand human behavior and interact with humans in more natural ways. However, such systems are limited by their reliance on hand-authored sociocultural knowledge and models. We introduce an approach to automatically learn robust, script-like sociocultural knowledge from crowdsourced narratives. Crowdsourcing, the use of anonymous human workers, provides an opportunity for rapidly acquir­ing a corpus of examples of situations that are highly specialized for our purpose yet sufficiently varied, from which we can learn a versatile script. We describe a semi-automated process by which we query human workers to write natural language narrative examples of a given situation and learn the set of events that can occur and the typical even ordering.